Skip to content

fix: handle null date and timestamp format arguments#23641

Merged
alamb merged 2 commits into
apache:mainfrom
lyne7-sc:fix/datetime-null-formats
Jul 17, 2026
Merged

fix: handle null date and timestamp format arguments#23641
alamb merged 2 commits into
apache:mainfrom
lyne7-sc:fix/datetime-null-formats

Conversation

@lyne7-sc

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

The formatted to_date and to_timestamp* functions can produce internal errors when the input string is NULL or when all format arguments are NULL.

A NULL value in a format column is also treated as an empty string instead of being skipped.

to_unixtime uses the same timestamp parsing path, so it is affected as well.

What changes are included in this PR?

  • Return NULL when the scalar input string is NULL.
  • Return NULL when all format arguments are NULL.
  • Skip NULL formats in array rows.
  • Keep returning a parsing error when all non-NULL formats fail.

Are these changes tested?

Yes, SLTs were added for to_date, to_timestamp, and to_unixtime.

Are there any user-facing changes?

Yes, only the bug fix described above. No API changes.

@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) functions Changes to functions implementation labels Jul 16, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 73.33333% with 4 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@3e058f0). Learn more about missing BASE report.

Files with missing lines Patch % Lines
datafusion/functions/src/datetime/common.rs 73.33% 2 Missing and 2 partials ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main   #23641   +/-   ##
=======================================
  Coverage        ?   80.65%           
=======================================
  Files           ?     1086           
  Lines           ?   366105           
  Branches        ?   366105           
=======================================
  Hits            ?   295279           
  Misses          ?    53230           
  Partials        ?    17596           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@alamb alamb left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @lyne7-sc and @Jefffrey

ColumnarValue::Scalar(scalar) => match scalar.try_as_str() {
Some(a) => {
let a = a.as_ref();
// ASK: Why do we trust `a` to be non-null at this point?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😆 classic

@alamb
alamb added this pull request to the merge queue Jul 17, 2026
Merged via the queue into apache:main with commit 8a37a07 Jul 17, 2026
37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

to_date, to_timestamp, and to_unixtime fail on NULL string and format arguments

4 participants